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ABSTRACT 

Traditionally, Spacecraft Flight Operations at the 
Jet Propulsion Laboratory (JPL) have been per- 
formed by teams of Spacecraft experts utilizing 
ground software designed specifically for the current 
mission. The Jet Propulsion Laboratory set out to 
reduce the cost of spacecraft mission operations by 
designing ground data processing software that 
could be used by multiple spacecraft missions, 
either sequentially or concurrently. The Space 
Flight Operations Center (SFOC) System was 
developed to provide the ground data system capa- 
bilities needed to monitor several spacecraft simul- 
taneously and provide enough flexibility to meet the 
specific needs of individual projects. 

The Magellan Spacecraft Team utilizes the SFOC 
hardware and software designed for engineering 
telemetry analysis, both real-time and non-real-time. 
The flexibility of the SFOC System has allowed the 
Spacecraft Team to integrate their own tools with 
SFOC tools to perform the tasks required to operate 
a spacecraft mission. This paper describes how the 
Magellan Spacecraft Team is utilizing the SFOC 
System in conjunction with their own software tools 
to perform the required tasks of spacecraft event 
monitoring as well as engineering data analysis and 
trending. 
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1. SFOC INTRODUCTION 

The Space Flight Operations Center (SFOC) is a 
collection of hardware and software subsystems 
working together in a multi-step process to provide 
mission control, telemetry processing, and storage 
services for a portion of the Jet Propulsion 
Laboratory’s (JPL’s) space flight missions (Ref. 1). 
SFOC receives spacecraft telemetry collected by the 
Deep Space Network (DSN) and performs frame 
synchronization and channelization of the telemetry. 


Once the telemetry is channelized, it is distributed 
to the user for real-time display and interpretation. 
This channelized telemetry is also archived to allow 
retrieval and analysis at a later date. All the basic 
tools necessary for a user to display real-time 
telemetry and retrieve archived data are provided by 
SFOC. 

2. MAGELLAN 

The Magellan Spacecraft is a National Aeronautics 
and Space Administration’s (NASA) planetary mis- 
sion designed to obtain high-resolution Synthetic 
Aperture Radar (SAR) images of Venus. Magellan 
was launched aboard the Space Shuttle Atlantis on 
May 4, 1989 and using the IUS, was placed in a 
type IV transfer orbit arriving at Venus on August 
10, 1990. Magellan began SAR imaging of the 
Venusian surface on September 15, 1990 and to date 
has produced high-resolution imagery of over 99% 
of the planet’s surface. 

3. MAGELLAN and SFOC 

SFOC was designed to be the first true multi- 
mission ground data system to operate at JPL. It 
supplies core ground data system capabilities with 
enough flexibility to adapt to specific mission needs 
in a timely and cost effective manner. The first user 
of SFOC was the Magellan mission. The Magellan 
flight team became involved early in SFOC develop- 
ment and helped to provide requirements, testing 
and feedback during the initial releases of each 
SFOC version. This led to the Magellan ground 
system configuration depicted in Figure 1. The first 
version of SFOC used by the Magellan spacecraft 
team was Version 6. This version was used to sup- 
port some of the pre-launch spacecraft testing and 
spacecraft team training exercises. The next ver- 
sion, Version 7, was used to support launch and 
the majority of cruise operations. Magellan is 
currently using Version 13, which came on-line just 
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prior to Venus orbit insertion. The latest versions of 
SFOC are being developed under the new Advanced 
Multi-Mission Operations System (AMMOS) pro- 
gram. 



Figure 1 Magellan Ground Data System 

4. SFOC REAL-TIME CAPABILITIES 

SFOC provides the following real-time telemetry 
monitoring capabilities: 

- Data Monitor and Display (DMD), 

- Derived telemetry channels, 

- Alarm limit maintenance and usage, 

- Latest available data dump requests, 

- Real-time memory readout verification. 

4.1. Data Monitor and Display (DMD) 

The Magellan spacecraft team uses the SFOC Data 
Monitor and Display (DMD) capability to monitor 
real-time telemetry via the spacecraft team worksta- 
tions. Each subsystem has customized its DMD 
environment based on the characteristics of the indi- 
vidual subsystem and personnel. The DMD func- 
tion allows engineers to define their own display 
screens or templates using the Template Definition 
Language (TDL). Although simplistic in capability, 
TDL provides the user with the tools required to 
create imaginative displays. The display types used 
by the Magellan spacecraft team include Fixed, List, 
Matrix, Message, Alarm and Plot pages. 

Fixed Pages: These display types are designed by 
the subsystem engineers to logically group data in 
pictorial form which best suits their monitoring 
needs. The "fixed" indicates that the formats and 
data assignments cannot be modified in real-time. 


This page type is preferred by the Magellan space- 
craft team for real-time data monitoring. An exam- 
ple of a Magellan power subsystem fixed display is 
depicted in Figure 2. The Magellan spacecraft team 
has designed and coded in excess of 100 fixed pages 
for flight use. 



Figure 2 DMD Fixed Display 

List/Matrix Pages: The data contents of list and 
matrix pages are designed to be modified in real- 
time simply by modifying a list of data channels. 
The data attributes to be displayed such as titles, 
values, time, units, etc are designed to be displayed 
on a single line for a list page and in a grid for 
matrix pages. This display type is useful when a 
subsystem wants to change data content on the fly. 

Message Pages: The message page is designed to 
emulate a line printer on a video display. As new 
channel messages appear, they scroll down the 
screen. The format of the message page is user 
definable allowing the individual to display data 
attributes of interest. Message page output can also 
be routed to printers or data files. 

Alarm Pages: This page type is designed to display 
channel information of telemetry channels which 
exceed a predefined threshold. The alarm page is 
structured like a list page. When a channel goes into 
alarm it is added to the alarm page and it maintains 
its position as data is updated. 

Plots: For subsystems that monitor analog data, the 
capability to plot telemetry real-time has been 
extremely valuable. Plot types include Channel vs. 
Time, Channel vs. Channel and Minimum/Maximum 
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plots. The DMD is capable of displaying 20 indivi- 
dual plot windows on a single workstation screen 
with up to four channel assignments per plot win- 
dow. This allows up to 80 channels to be monitored 
graphically. The size of the plots, scales, labels and 
time ranges are all user definable. 

Figure 3 shows a typical subsystem display 
configuration and contains fixed, list, alarm, and plot 
display types. 

4.2. Derived Telemetry Channels 

The DMD also provides the capability, via the 
Channel Conversion Language (CCL), to produce 
derived telemetry channels. Derived telemetry 
channels are measurements which are derived from 
information received in the main telemetry stream. 
The derived channels can be as simple as changing 
the units on a received value, to as complex as 
deriving the remaining propellant load on a space- 
craft In general, any data that is used for spacecraft 
analysis that can be generated from a snapshot of 
telemetry is a likely candidate for a derived channel. 
Just like the main telemetry channels, derived chan- 
nels can be displayed, alarm checked, and plotted 
using the DMD. They can also be retrieved from 
the central database for non-real-time analysis or 
trending. The calculations of "Right Ascension and 
Declination" are simple but useful examples of 
derived channels developed by the Magellan space- 
craft team. The algorithm for calculating the right 
ascension and declination utilizes the 4 quaternion 
elements telemetered from the spacecraft and is as 
follows: 

c 31 = 2 x (( quat 1 x quat 3) + (quat 2 x quatA)) 

c 32 = 2 x ((quat 2 x quat 3) - (quat 1 x quat 4)) 

c 33 = quatA 2 + quat 3 2 - quat 2 2 - quat l 2 

right ascension = AT AN (c32,c31) 
declination = ASIN (c33) 

The Magellan flight team has developed and imple- 
mented approximately 250 derived channels during 
mission operations. 

4 3. Alarm Limit Maintenance and Usage 

The capability exists within the DMD to check user- 
specified telemetry and inform the user when an 
unexpected value is encountered. This function is 
accomplished using DMD yellow and red alarms. 
The yellow alarms are set and monitored by subsys- 
tem engineers locally at each individual workstation 
and are generally used as caution alarms. It requires 
only subsystem lead approval to change yellow 


alarms. The red alarms are monitored on a system 
wide basis and usually indicate that an 
undesirable/take-action event has occurred. They 
are monitored round-the-clock by Magellan mission 
control team specialists. Red alarm values have 
been typically selected utilizing the following guide- 
lines: 

- Adherence to launch/deploy go/no-go criteria, 

- Adherence to specified acceptance test limits, 

- Adherence to S/C fault protection thresholds, 

- Predicted values for mission phases, 

- Nominal S/C values observed pre-launch. 

To insure the correct level of review. Team Chief 
level approval is required to change a red alarm 
value. 

4.4. Latest Available Data Dump Requests 

Another feature of DMD used by the spacecraft 
team is Latest Available Data (LAD) dump requests. 
Maintained in the workstation is a buffer which con- 
tains the latest available data of all activated 
telemetry channels. The contents of the buffer may, 
at any time, be dumped to a printer or a file for 
further processing by spacecraft analysis tools. The 
Magellan spacecraft team used this capability to 
capture the state of the spacecraft following an ano- 
maly which caused unexpected loss of downlink. 
One of the first actions performed after the anomaly 
occurred was a dump request of the LAD buffer. 
This LAD dump was immediately reviewed by the 
subsystem engineers for clues to the cause of the 
anomaly. 

In addition to LAD dumps, the capability to perform 
channel parameter table dumps and coefficient 
dumps is also available. These capabilities can be 
used to determine channel information such as 
current alarm limits or polynomial coefficients. 

4.5. Real-time memory readout Verification 

There have been several instances during the mis- 
sion where the spacecraft team has had to make 
flight software modifications or parameter changes 
that require real-time Memory ReadOut (MRO) 
verification. The majority of these software 
modifications are performed by a sequence of com- 
mand groups. In general, memory verification of one 
command group is required before the next com- 
mand group may be sent. This verification was usu- 
ally required in a time-critical manner. To allow 
timely MRO verification, SFOC developed a tool to 
perform real-time MRO verification. This tool, 
named BCTOMRO, strips MRO data from the 


723 



Figure 3 Typical Magellan Subsystem Display Configuration (Power Subsystem Battery Recondition Display) 

























SFOC telemetry stream and displays the data in 
real-time. BCTQMRO has allowed the spacecraft 
team to make the real-time MRO verifications of 
changes and continue commanding during time criti- 
cal command windows. 

5. NON-REAL-TIME DATA ANALYSIS 

SFOC has provided several tools to aid in non-real- 
time analysis of spacecraft data. The spacecraft 
team uses these tools in combination with tools they 
developed to perform the following non-real-time 
functions: 

- Telemetry recall from the central database, 

- Automated analysis and trending, 

- Telemetry state verification, 

- MRO data recall from the central database, 

- Analysis of spooler files. 

5.1. Telemetry Recall from the Central Database 

The most important SFOC tool developed for data 
analysis on the Magellan program has been the 
capability to query data from the Central DataBase 
(CDB). The Magellan CDB consists of 45 days of 
on-line storage of channelized spacecraft engineer- 
ing telemetry, DSN monitor data and associated 
ancillary data channels. The query capability has 
been set up so that any engineer from a SFOC- 
provided workstation can extract data concerning the 
spacecraft or ground activities. The SFOC-provided 
tools used by the spacecraft team for querying the 
CDB include DMDETF, DMDQMOD, and 
DMDQUERY. These tools allow a user to produce 
"canned" queries that are used repetitively or "ad- 
hoc” queries that are built and submitted in minutes. 
The user selects the desired telemetry channels and 
the analysis interval. The data is returned in an 
ASCII Data Return File (DRF). This DRF can then 
be input to any one of numerous analysis programs 
or may be used as a stand-alone tool for data 
analysis. 

5.2. Automated Analysis and Trending 

Although the CDB query capability itself has been 
extremely useful, it became apparent that the Magel- 
lan spacecraft team analysts were performing a 
variety of repetitive queries and analysis during their 
daily and weekly routines. To make better use of an 
analyst’s time these repetitive tasks have been 
automated using Unix-based scripts. This automation 
allows the analyst to perform additional mission 
analysis and anomaly resolution. The majority of 
the automation tools consists of scripts written in 


AW K, PERL, CSH, SH, etc, ... and some "C" code. 
The automated tasks were derived from manual 
tasks allocated to each subsystem in flight team pro- 
cedures. The tasks were performed manually until 
they were well understood and then the engineers 
constructed a tool to carry out the task. It is impor- 
tant to note that the subsystem engineers performed 
their own coding since the spacecraft team did not 
have a dedicated software staff. One example of an 
automated script is the engineering telemetry long- 
term trending script The first step in this script is 
to query the desired analog telemetry (temperatures, 
voltages, currents, etc.) over a specified interval, 
usually 24 hours. The script uses the resultant DRF 
to calculate the minimum, maximum, average and 
standard deviation for each telemetry channel over 
in the entire length of the file. These statistics are 
then appended, along with a time stamp, to a set of 
long-term channel trend files. There is a channel 
trend file for every telemetry channel of interest. 
The result is a set of long-term trend files that con- 
tain trending statistics for each 24 hour period from 
the beginning of the mission for each channel of 
interest. These trend files are formatted like DRFs 
which allows them to be plotted and analyzed using 
the same tools used to plot and analyze DRFs. 
Therefore, the trend files are used for analysis and 
are periodically plotted against long-term predictions 
using VMPLOT. An example would be to plot the 
actual long-term solar panel maximum current trend 
versus predicted solar panel current to quantify 
panel degradation over the life of the mission. The 
process outlined above has been fully automated 
with use of "perpetual” scripts. A "perpetual" script 
performs its task and then re-submits itself to run 
again at the next specified interval, without any 
operator intervention. For instance, once a day an 
automated script queries the desired channels to be 
trended and re-submits itself to run the next day. 
One other advantage of the automation scripts is 
their ability to run during early morning hours when 
system loading is low. 

5.3. Telemetry State Verification 

Originally Magellan did not have the capability to 
autonomously track spacecraft states. All critical 
component state tracking was accomplished by 
engineers from each subsystem manually recording 
their individual subsystem states in a spacecraft state 
table. However, soon after launch, the spacecraft 
team developed the ability to autonomously extract 
all spacecraft states that are telemetered. The 
spacecraft team created a Unix script entitled 
’’Moose Query" which queries all Magellan 
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telemetry and produces a summary report of all 
spacecraft states. Although this report is still com- 
pared manually to a table of expected spacecraft 
states, it greatly simplifies collection and recording 
of the approximately 1000 spacecraft states. 

In addition to the state reports, moose query also 
produces plots of all analog data and tabular prin- 
touts of all status information over a user selectable 
time range. Since all spacecraft telemetry is either 
plotted or printed, this tool proved valuable during 
anomaly investigations. Following critical 
anomalies, a moose query would be submitted and 
the engineers would receive a copy of the output 
within hours for review. It should be noted that 
many anomalies were resolved by data from 
telemetry channels showing anomalous indications 
but were not immediately thought to be related to 
the anomaly. 

5.4. MRO Data Recall from the Central Data- 
base 

In addition to storing engineering telemetry, the 
CDB also has the capability to store MRO data for 
retrieval at a later date. The MRO query utility 
allows the engineer to retrieve memory readouts 
received from the spacecraft for any of the on-board 
computer memories. The user supplies the desired 
memory device, the start and end addresses, and the 
begin and end times of MRO collection. The MRO 
utility queries the CDB and returns an ASCII file 
identifying address and the memory contents at that 
address. This return file is utilized by the spacecraft 
team analysis programs, MEMANAL, VMPTRK, 
CHKSUM, and VMFLOAD to verify the contents of 
on-board memory and to verify flight software 
parameter values. Once the MRO is verified to be 
correct, the file is catalogued as the current updated 
ground memory mask. This mask is used as a base- 
line if an on-board computer memory ever needs to 
be reloaded. This process was used to reload AACS 
"Memory B" after an address bit failed in one 
memory bank shortly after Venus orbit insertion. 

5.5. Analysis of Spooler Files 

SFOC can also produce spooler files of telemetry 
data. These spooler files represent a collection or a 
stream of telemetry for a selected time period. The 
files are expanded to include all telemetry and ancil- 
lary information such as header data, monitor data, 
quality flags, etc. The processing of these spooler 
files is most easily done using a SFOC-provided 
utility called BROWSER. BROWSER allows an 
engineer to review spooler files and analyze the con- 
tent for correctness. BROWSER has the capability 


to build templates that a) display data in a certain 
orientation, or b) filter through a spooler file 
displaying only the data that match predefined con- 
ditions. The capability to produce and analyze 
spooler files has proven useful for the Magellan 
flight team when troubleshooting problems with the 
telemetry stream. For example, the use of spooler 
files and BROWSER allowed spacecraft analysts to 
isolate and provide solutions to a radar telemetry 
spiral wrap anomaly. A "raw" spooler file and a 
"channelized" spooler of the same data were com- 
pared using BROWSER. From this analysis it was 
concluded that the ground decommutation process 
was working correcdy and the problem was isolated 
to a timing problem within the radar, which has 
since been corrected. BROWSER was also used 
extensively to isolate the on-board tape recorder 
problems. 

6. SUMMARY 

SFOC has provided some extremely powerful 
analysis tools. The DMD provides enough flexibil- 
ity to meet the dynamic real-time analysis needs of 
each individual subsystem and, although considered 
"Expert Friendly", it does not constrain the user as 
do many sophisticated graphical user interfaces. 
The capabilities and flexibility of the CDB query 
process have significantly contributed to the success 
of the Magellan mission. The versatility and flexibil- 
ity of the SFOC tools and Unix have allowed the 
spacecraft team to build software and scripts "on top 
of SFOC tools to provide automation and remove 
manual burdens from the analysts. Although the 
SFOC tools experienced some growing pains, the 
SFOC development staff has always responded well 
to problems and suggestions. Being involved in the 
development and use of SFOC has been a rewarding 
experience. 

7. REFERENCES 

1. Miller, D„ Palkovic, L., Edwards, C., "Space 
Flight Operations Center User’s Overview", July 
1991, pages 1-6. 

8. ACKNOWLEDGEMENT 

The work described in this paper was accomplished 
by Martin Marietta under contract to the Jet Propul- 
sion Laboratory, California Institute of Technology 
and sponsored by the National Aeronautics and 
Space Administration. 


726 



